1) Field of the Invention
The present invention relates to an apparatus comprising a plurality of modules connected to one another through interface buses such as PCI (Peripheral Component Interconnect) buses or the like, and having a function of confirming data transfer among these modules. More concretely, the present invention relates to a storage controlling apparatus disposed between a physical device (such as a magnetic disk unit) and a host to control an access from the host to the physical device, and interface modules (such as a host interface module, a disk interface module and the like) for use in the storage controlling apparatus.
2) Description of the Related Art
FIG. 4 is a block diagram showing structures of a storage apparatus (a storage controlling apparatus) and for the description of the related art, FIG. 4 is discussed with respect to known structures of the storage apparatus. A storage apparatus 1 shown in FIG. 4 writes data from a server (a host) 4, and reads data requested by the server 4 and transfers the data to the server 4 in response to an access from the server (host) 4.
The storage apparatus 1 comprises a disk enclosure 2 having a plurality of disk units (physical devices) 2a, and a storage controlling apparatus 3 disposed between the disk units 2a and the server 4 to control an access to each of the disk units 2a from the server 4.
The storage controlling apparatus 3 comprises a disk interface module 10, a host interface module 20, a management module 30 and a PCI bridge module 40.
The disk interface module 10 controls an interface (data transfer) with each of the disk units 2a in the disk enclosure 2 through a disk interface bus 54.
The host interface module 20 controls an interface (data transfer) with the server 4 through a fiber channel interface bus 50.
The management module 30 generally manages the whole of the storage controlling apparatus 3. The management module 30 is equipped with a cache memory temporarily storing data to be written from the server 4 into one of the disk units 2a or data to be read out from one of the disk units 2a to the server 4, and manages the cache memory.
To the PCI bridge module 40, the disk interface module 10, the host interface module 20 and the management module 30 are connected through the PCI buses (interface buses) 51, 52 and 53. The PCI bridge module 40 connects the disk interface module 10, the host interface module 20 and the management module 30 to one another so that data can be transferred among them.
With the above structure, data writing from the server 4 into one of the disk units 2a and data reading from one of the disk units 2a to the server 4 are performed as follows.
When data is written from the server 4 into one of the disk units 2a of the disk enclosure 2, data to be written is transferred from the server 4 to the host interface module 20 through the fiber channel interface bus 50, then temporarily stored in the cache memory of the management module 30 via the PCI bridge module 40 through the PCI buses 52 and 53, then stored in the cache memory (refer to an arrow A1 in FIG. 4). After that, the data to be written stored in the cache memory of the management module 30 is transferred to the disk interface module 10 via the PCI bridge module 40 through the PCI buses 53 and 51, and written in the predetermined disk unit 2a from the disk interface module 10 through the disk interface bus 54 (refer to an arrow A2 in FIG. 4).
Conversely, when data is read out from one of the disk units 2a of the disk enclosure 2 to the server 4, data to be read is transferred from the disk unit 2a retaining the data to the disk interface module 10 through the disk interface bus 54, then temporarily stored from the disk interface module 10 in the cash memory of the management module 30 via the PCI bridge module 40 through the PCI buses 51 and 53 (refer to an arrow A3 in FIG. 4). After that, the data to be read stored in the cache memory of the management module 30 is transferred to the host interface module 20 via the PCI bridge module 40 through the PCI buses 53 and 52, then read out to the server 4 from the host interface module 20 through the fiber channel interface bus 50 (refer to an arrow A4 in FIG. 4).
Next, description will be made of structures of the disk interface module 10 and the host interface module 20 in the storage controlling apparatus 3 shown in FIG. 4, with reference to a block diagram shown in FIG. 5. The disk interface module 10 and the host interface module 20 have basically the same structure. However, the host interface module 20 differs from the disk interface module 10 in that the host interface module 20 has a function of converting an optical signal from the fiber channel interface bus 50 into an electric signal, and a function of converting an electric signal in the host interface module 10 into an optical signal and sending the optical signal to the fiber channel interface bus 50, while the disk interface module 10 does not have these functions.
As shown in FIG. 5, each of the disk interface module 10 and the host interface module 20 (hereinafter, simply referred as the interface modules 10 and 20 occasionally) comprises a CPU 11, a chip set 12, a memory 13, an interface module-LSI 14, a data buffer 15 and a fiber channel chip 16.
The CPU (Central Processing Unit) 11 functions as a first processing unit generally managing the interface module 10 or 20.
The chip set 12 has a function of connecting the CPU 11 to another device (for example, the memory 13) and a function of connecting the CPU 11 to the PCI bus 17. The CPU 11 is connected to the memory 13 via the chip set 12, and is also connected to the interface module-LSI 14 via the chip set 12 through the PCI bus 17. The memory 13 is configured with, for example, an SDRAM (Synchronous Dynamic Random Access Memory).
The interface module-LSI (Large Scale Integration) 14 functions as a second processing unit (a second transfer processing unit) controlling data transfer between the interface module 10 or 20, and the PCI bridge module 40 through the PCI bus 51 or 52. The interface module-LSI 14 is provided with the data buffer 15.
The data buffer 15 is configured with, for example, a DDR (Double Data Rate)-SDRAM, for example. The data buffer 15 temporarily stores data to be transferred to the PCI bridge module 40 and data transferred from the PCI bridge module 40.
To the interface module-LSI 14, the PCI bridge module 40 is connected through the PCI bus 51 or 52, the chip set 12 is connected through the PCI bus (interface bus) 17, and the fiber channel chip 16 is connected through the PCI bus (interface bus) 18.
The interface module-LSI 14 has a function being as a DMAC (Direct Memory Access Controller; controlling means). As will be described later with reference to FIGS. 6 and 7, the CPU 11 normally runs a predetermined control program to set a descriptor in the DMAC in the interface module-LSI 14, thereby to make the interface module-LSI 14 execute data transfer or reading of a confirmation code (AA code).
The fiber channel chip (FC-Chip) 16 functions as a first transfer processing unit controlling data transfer between the interface module 10 or 20, and each of the disk units 2a or the server 4 through the fiber channel interface bus 50 or the disk interface bus 54.
When the interface module 10 or 20 transfers data to the cache memory of the management module 30, the data is transferred via the PCI bridge module 40 through the PCI buses 51 or 52, and 53 as described above with reference to FIG. 4. At this time, the interface module 10 or 20 can confirm whether the data is normally transferred to the PCI bridge module 40 through the PCI bus 51 or 52, but cannot confirm whether the data is normally transferred from the PCI bridge module 40 to the management module 30. Since this is a problem in a system required reliability, it is necessary that the interface module 10 or 20 can confirm whether the data is normally transferred from the PCI bridge module 40 to the management module 30.
In Japanese Patent Laid-Open Publication No 2001-243206, there is proposed a technique of performing normal end confirmation (AA: Access Assurance) of PCI-transfer from the PCI bridge module 40 to the management module 30. In order to realize this technique, the PCI bridge module 40 has a function being as a confirmation code setting means 41 (refer to FIG. 4). The confirmation code setting means 41 sets, in the PCI bridge module 40, a confirmation code (AA code) for confirming data transfer from the PCI bridge module 40 to the management module 30 when the interface module 10 or 20 transfers data to the management module 30 via the PCI bridge module 40 through the PCI buses 51 or 52, and 53. The interface, module 10 or 20 determines whether or not the data transfer from the PCI bridge module 40 to the management module 30 has been normally carried out by reading out the above confirmation code from the PCI bridge module 40 after the data transfer, and carries out a process (normal end or data re-transfer) according to a result of the determination.
Hereinafter, the above technique will be described in more detail with reference to FIGS. 6 and 7.
First, description will be made of an operation at the time of normal data transfer of the storage controlling apparatus 3 described above, with reference to a sequence diagram (steps S11 through S35) shown in FIG. 6. When staring data transfer, the CPU 11 sets, in the interface module-LSI 14 (DMAC), a descriptor for data transfer and a descriptor for confirmation code reading (data transfer confirmation descriptor) for each data unit (data block) to be transferred (set DSCs: step S11), and starts the DMA (step S12).
Here, the descriptor (DSC) has a set of information required at the time of DMA transfer. The descriptor for data transfer contains transfer information (for example, transfer length, transfer origin address, transfer destination address, and other additional information) required for data transfer to the management module 30. The descriptor for confirmation code reading contains confirmation code reading information (for example, transfer length, transfer origin address, transfer destination address and other additional information) required to read the confirmation code from the PCI bridge module 40 to the interface module 10 or 20. FIGS. 6 and 7 show an example in which three data units (data 1, 2 and 3) are successively transferred. In this case, it is necessary to set three descriptors for data transfer and three descriptors for confirmation code reading at step S11.
The interface module-LSI 14 in which the descriptors are set and the DMA is started by the CPU 11 is loaded the descriptor for data transfer for each data unit (load DSC; steps S13, S19 and S25). The data is transferred from the interface module-LSI 14 to the management module 30 via the PCI bridge module 40 according to the loaded descriptor for data transfer (steps S14, S15, S20, S21, S26 and S27). When the data transfer is completed, the descriptor for confirmation code reading is successively loaded (step S16, S22 and S28). The confirmation code (AA codes 1, 2 or 3) for the data is obtained from the PCI bridge module 40 according to the loaded descriptor for confirmation code reading (steps S17, S23 and S29) and stored in the data buffer 15 (steps S18, S24 and S30)
When transfer of the three data units is completed, an end notice (interruption) is sent from the interface module-LSI 14 to the CPU 11 (step S31). When receiving the end notice, the CPU 11 reads out a status from the interface module-LSI 14 (step S32), and determines on the basis of the status whether the data transfer between the interface module 10 or 20 and the PCI bridge module 40 has been carried out normally or abnormally (step S33). When it is determined at step S32 that the data transfer has been carried out normally, the CPU 11 reads out the confirmation codes from the interface module-LSI 14 (data buffer 15) (step S34), and determines on the basis of the confirmation codes whether the data transfer between the PCI bridge module 40 and the management module 30 has been carried out normally or abnormally (step S35). The sequence diagram shown in FIG. 6 shows an operation of the storage controlling apparatus 3 performed when the data transfer has been carried out normally. At step S35, it is determined that the data transfer has been carried out normally, and a series of data transfer is completed.
Next, description will be made of an operation of the storage controlling apparatus 3 performed when abnormality occurs, with reference to a sequence diagram (steps S11 through S34, S36, S37, S11′, S12′, and S19′ through S35′) shown in FIG. 7. In FIG. 7, at steps designated by like reference characters, like or corresponding processes are carried out, detailed description of which are thus omitted. In FIG. 7, “′” is attached to the reference character of a step at which data is re-transferred.
In an example shown in FIG. 7, three data units are transferred and confirmation codes are obtained in a procedure (steps S11 through S34) similar to that shown in FIG. 6. However, in the example shown in FIG. 7, a transfer error occurs between the PCI bridge module 40 and the management module 30 when the second data 2 is transferred, and the transfer of the data 2 is abnormally terminated.
When determining on the basis of a confirmation code read out from the interface module-LSI 14 (data buffer 15) at step S34 that the data transfer has been carried out abnormally, the CPU 11 detects contents of the error from the confirmation code (AA code 2) (step S36), confirms a transfer error of the data 2, and executes re-transfer of data units after the second and the later data units (here, two data 2 and 3) (step S37).
When re-transferring, the CPU 11 again sets, in the interface module-LSI 14 (DMAC), descriptors for data transfer and descriptors for confirmation code reading for the two data 2 and 3 to be re-transferred (step S11′), and starts the DMA (step S12′). Like the procedure (steps S19 through S30) shown in FIG. 6, the data 2 and 3 are transferred and the confirmation codes (AA codes 2 and 3) are obtained and stored (steps S19′ through S30′).
When re-transfer of the two data units is completed as above, a process (end notice, reading of a status and confirmation codes, determination of normality/abnormality; steps S31′ through S35′) similar to the process at steps S31 through S35 is executed. When it is determined at step S35′ that the data transfer has been carried out normally, a series of data transfer is completed.
In the technique shown in FIGS. 6 and 7, the CPU 11 has to set two kinds of descriptors (for data transfer and for confirmation code reading) for each data unit to be transferred, and execute both determinations of whether data has been transferred normally or abnormally between the interface module 10 or 20 and the PCI bridge module 40, and whether data has been transferred normally or abnormally between the PCI bridge module 40 and the management module 30.
With an increase in data transfer quantity from the host (server) 4 in these years, it is a large load on the CPU 11 to set a descriptor for confirmation code reading for each data unit, thus a PCI access takes a longer time. Additionally, the process of setting the descriptor for confirmation code reading becomes a large overhead, which causes a degradation of the input/output performance.
The CPU 11 executes all the normal/abnormal determinations, which is a large load on the CPU 11. For this, the PCI access takes a longer time, thus the determining process becomes a large overhead, which results in degradation of the input/output performance.